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. - The document presents a formative evaluation of 13° 
Canadian studies kits being developed in Alberta, Canada. The 
abjective was to ascertain the strengths and wéaknesses of the kits . °. 
as well as to determine whethet the program development model was - , 
effective in producing quality materials. Quality was defined in a 
terms of internal consistency, inter-unit consistency, program . 
consistency, external consistency, and significance. Questionnaires: 
-werg mailed to teachers und students in 74 K-12 pilot classes; 1488 
were returned. In the area ‘of internal consistency, questions 
relating to focus and explicit activity sequances and objectives were 
raised. Inter~ unit consistency, especially in skill development, was 
~ almost impossible given the process used to develop the kits. - 
Questions raised relating to program consistency were aimed at = 
' refining congruency with provancial handbooks. External consistency | 
examined congruency with current social studies practice. Questions 
were raised relating to,all-16 characteristics of sound programs. Ail 
respondents reacted positively to the significance of the Canadian 
content. Questions raised concerned a logical overall plan ,for 
content coverage in all the kits and avoidance of overlap and 
imbalance. .Possible recommendations included hiring overall, editors, 
developing a series of "How to Do It" booklets, finding resolutions 
to consistency probleas, anstituting quality control, and presenting. 
content summary charts in each kit. Appendices present the program a 
analysis form and student questionnaires. (CK) . . 
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Preface To The Study o ¢ ‘ 


The approach paken in this study is based da ta ‘certain ‘premises 


concerning formative evaluation, “ 


Vv ar - .  % - ie a 

Curriculum development is a task which requires much time, energy, 

‘ . 7 ot & : , ea ! 
resources ‘and expertise. as a regult, developers ‘tend to become committed to’ 


my 


( 
the wor thuhileness apd eelaveces of the programs eels. are produced. Egos 


*- become an ‘integral. part of program deve lopment eucdneke: It may become 
aa difficult, therefore, for ‘ndivtduals . detach themselves from their work ah : 


and to critically reflect upon what they have done. 
w ; ‘ U an x 


. The’ task of formative evaluation is to help foster this reflective - 
attitude while respecting the developer's commitment and ego involvement in 


what they have developed., . _ 2 | . i 
. . 1 ‘. . & ; ; ; 
One way to accomplish this task is to pose questions to the 
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developers rather than to ake judgmental statements, Soaut tors tend to ° 

_ invite responses which often clarify the issue and which allow consensus at 

at! | atnong developers. On ne other ‘hand, -Jédgmental ethpements tend to elicit: 
| reaction and defense. The purposes -of formative ay ancien in thie project _ 


.' were better ‘served through questions. ||). | ne ee 


/ = : . 7 | coe lo | a 
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: The aim of this evaluative study was to present the program c 
developers with questions for discussion purposes, Twe legitimacy and. : 


e . 
«relevance of the qugastions posed ultimat®ly rests with the team members...’ | 
+ ; ‘: ’ ‘ . he car 


Scope of the Study oo 


Purpose ; : ~ : é . : . _— g ; a e 


‘This study was designed to be a formative evaluation of ‘the thirteen: 


Canada studies kits currently being developed within. Alberta. : Assessment rem. 
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ports wete ‘Provided to the development Keane for use in possible revisions 


of disseaiee. Information within these reports: was. organized around the F 


following areas: ae . ss ee ke 
5 a ‘ e soe 

1.0 Strengths of the Kit, - 
o- 2.0 Concerns about the Kit A ae { an 2 Z. 


‘ 2.1 “Concerns raised by Pilot Students . 


2.2 Concerns raised by Pilot Teachers 


oad 


2.3 Concerns ‘about Tatetnal Consiarency 


‘ | 2.4 Concerns about Consistenc ith Cuyrent Social 
Pe Studies Practice , , as 


2.5 Concerns about Congruency with the Alpdx'ta Social 
Studies Program © 


2.6 “Concerns about the Significance “of the Manadian 
Content Selected 


acai for Geng idveaeign . 
; ; . s on : 
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Procedures — ‘ \ 
———— a“ 
TA nyaeer of IHS EECRSHES: 9 for obtaining systematic. information con* 


cerning each of the thirteen kits were designéd and used in the study: 
1. Ittstrument for content: analysis of each kit (AppendixT). 


2. Instrument for ebisiaing pilot teacher: ASgeeamrTe of the 
kits (Appendix "ay. 7’ : 


i] 
) 3. Instrum&gts for obtaining pilot student assessment of the 
. kits (Appendix Ty). } 


“Data epeaencd by means of these instruments were..uged in the compilation of 


a report on each’ kit (appendix 1). 


- opment) of the unite. 


1977 to, June YQ, We 2 eae =. 7 
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Bach of the’ chirteen reports was presented 
Ne 


; arid discussed: with the appropriate team who had been n Kesponsible for the. devel- * _ 


S.- 
The study was completed over a six month period (January, 4, 
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‘Though sufficient questionnaires were mailed hal every teacher and 


aeusent involved in thé pilot classes the thirteen kite, the returns varied a 


chew 


How effective has the process of program development been in producing ; 


quality social studies matérials? Quality was defined in terms/of the following —— 
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the intents, activities, and content. ‘ 
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In the majority of the kits questions were posed about. internal con- 
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‘ sources, and strategies of each. kit? The units were examined for link 


ea among, : 


“questighs: bs 7 oo YY 
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Internal. Coisistenc . of a" G4. 8 
‘se | Is there consietency between and among. the objectives, content, re- 


_ sistency. Key coricery were. raised.to help in refining these linkages: 
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‘acrogs | the grade levels: : ae eee a es. } 
me SUMMARY OF PILOT CLASSES ' es 
a Canadian Opntent: Kite for each Grade 
1f{2 [3 [4 [5 [6 |7 [8 19 Jrme 10] 111.12 | Totals 
» ae (ae 
Total Number of | | | , ols : 
Pilot-Classes * Se 12 (epee ye et at ite ie L. Me 
Pilot-Teacher ; 5 4 5 4 3 ‘ 8 ] 6 "9. 7 aso 4‘ 3 ‘ 63 
ne turns P : 
EL LOEerucene 101 {108 |119 [107 | 60 {185 | 96 [148 172 4141 | 48 116] 24] 1425 
Returns . aN : oe cr 
. 
| . ; | ; 7 | 
uestions Por The Study . ~~ ee ne eee 
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- 1... Should’ there be, an overall value issue to provide focus’ 


ee a , 4 _ _ P . ‘ _ 
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2. Should a master chart be prepawed showing where each ° 

r yO we : ee is —— in the kit? ee a ae oS 

; 7 . ae - 
Be ‘Should a wechanten that would help students ‘and, teachers 


maintain focus on the: central Enews of cha ete be ‘devised! os, 


“Ms 
“ : : ; : 
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7 4. Should the activity sequences designed to es con-, 
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+ “Before the Kite are » finally produced and disteibuted: they should’ be res . ; 


cepts/generalizations be made more expltett for teachers?, 
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examined to determine to what extent ‘these questions have been see 
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‘Inter-unit Consistency, | : . - oe 
: . “ ‘ : : : . 


‘Is there a —— and sequential consistency among the its for ~ 


v 


Cs the various grades? The, le itimacy on raising ‘this iesue is questionable, 
§ | 


a? 


~~ -., because the procesa used: to deverer the kits did little to ensure « ‘cross- # | ee 
oe . ; 
grade consistency. Time needed for detailed articulation among tHe kits” was 


not prevades to”. the deve lopment teams, snd Enere was no agreed upon master 


+plan or framework for ‘skills, conceptg, and iatgaax For exaiple, ett) : 
a 7 dee lopment in social studies programs ip usually based on the following. - | 
principles: % ys | 
i . (a)” Skills are best taught tient in the context of 
- : : 7 be —__—- —=n . e : <. : : - 
ea unit. . 
| oa 
Ps ~ 
_ ; — (b) The program of ifistruction should be flexible enough. . 
\ to allow alt 11s to be taught as they are needed by | 
~learners. | 7" 
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. prinetples (a) 1.0) nd (c) to operate, ‘but did. not acconinodate (d) and (e). 
y 
Questions were vatoed te ach kit. suggesting that, akill sequences be nade 


explicit. ' ° -  * ——- 


deseription of ekille. Rather’, the balance, comprehensiveness, and sequencing ” 


- about ‘the vagueness 0} or the “taken for gratited aur: ia 


an opportunity. lost th the peodeers Such a plan may’ have provitted not - only 


“acope ane sequence to the overall Ganada Studies Progtee, but also given direg- 
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es, a ye | 
, objectives: arid sequences “hiplict for teachers and students. a * 4 
et: — . ok . : a 
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~ - (e) ‘the. learner should understand ‘the purpose of the ek . 


© es and have need for developing: it. 
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ne oe . ae ‘, 
| (dy Skill development is most ‘effective when there ‘is. sys~ 
tematic and. continuous application. * . us - 
(e) . Skill instruction should be presentedCat increasing 
levels of difficulty-across the gradés,. | 
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The manner err the Canadian Content kits were developed a} lowed dx 
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. ae 
rae in yaten skill. epaecere were stated aud daveresat within 
Ehe units makes it victually ‘pmpossible to provide a meaningful crose-grade 
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of skills tends to be aad across the thirteen kits. Questions were ,raised - 
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the jack of a,master plan_for. the developmignt: of specific skills i was 
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“a the. ‘individual development teams. As showa by the following charts, 


estions diecussed with many of ehiene teame were addressed to making” skill 


"To what. degree are the: kites congruent . with the intents aie the AIDSEES - 


social Studies aa Criteria were. selected from R esponding to change. and 
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QUESTIONS POSED: | 1 


oe nos kit contribute to 
the elie oment of, signifi- [*: 
i cant socialy nquiry skills? - 


. Are specific skill develop! < 
“ment sequences included? 


sequential development of. 


skills? —_ ca ee 
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an, research methéds: included » 
in the skill objectibest 
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KITS FOR WHICH QUESTIONS WERE -RAISED CONCERNING 
SKILL DEVELOPMENT SEQUENCES | , . 
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| apnoreintkios: to utilize-strategies and materials thar are. supported by: current 


= informed opinion of what~constitutes good Peaeence ‘in. the peacnig of social 


mer 


‘ broadened. Further, teachers were overwhelmingly « favour able in their comments . 


Se 2 


: units, For, all of the kite the gaeectoue. rdised Were aimed at ‘refining con- 


Significance | . ‘e = ‘ 7 : = - | e = a ate 


a "Students reacted () fitively to most of the kits. They. felt that they learned 


| regarding’ having Canadian content avai lable for classroom USE. | They generally’ oa 


: perceived eee as being worthwhile and relevant for atudents. | However, on 


Beperfencés. in | Dectaton Making, “ana were. addr esaad fas quesfiions to each: of ra as 


\ 
‘gruency with the provincial handbooks. "The steel questions ‘posed can. be. 


found’ ‘in the. reports in Appendix 4, The fol loving chaxt provides: a summary 


of those units for which- enon eeene were raised concerning their: congruency with, 


“<The vest tons: eased ox ee analysis was: ‘ "Do the kits provide 


studies?" . Criteria for this assessment . were taken feod up- -dated characteris, - oe, 


tics identified originally by - Se C. NeLendon and F, Penix What Research - Says | | 
to_the Teacher: Teaching the Social Studies. National. Education Association, a 


* : 

1968.. Current literature on social studies practice wad deed in seleckitg these woo 
characteristics. For each of ‘thé ‘hhee “questions were raiséd. on the basis. of . | 
"5 a Ne 

these selected criteria. A summary is provided in the following chart. 7 a 
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How significant is the Canadian ania selected within the kits? — ee 5 + 
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tings hich they ‘didn't know before, aha that. their. knowledge of Canada was 


« é vena, 7 
& 
a) 


‘ 
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the Anitial selection of the content presented developers of ‘the. thirteen 


units with gome problens: 7 | gt eg ee a Be ae 


6 : : Sy! 


: Me . _ “ ? - ; . as . | ee - . a: 
a ” a7 “eo a: . ssa 8 eg 
: ale ‘Yow gould ‘the kit developers working independently of oe \ 
ee te _ ghee tana sone that. each. kit wa td form: a opical. 7 


mg? 


“Dees it focus on a perti- 
nent’ value issue? « 
° a a * t 


.Does it have future use 
‘for learners’, ~~ . 


, Does ft help learners under- 
stand a significant social 
problem? —_ 


> Is the kit Fa isvant ae 
’ needs and interests of 
learners? ; oy 
ee, Does itt contribute to the 
re development of. significant 
social/inquiry skills? 


Have. the concegts outlined 
in, the kit been developed?, 


Os Do thé concepts: utliffed form ; 


the bases. of the generali- 
oo zations developed? _ 


* Are specific skt11 deve lop- 
any 


‘Is the primary focus of. the, 7 


_. kit value inquiry?” 


.e : 


: om 


nt sequences included? |” 


* CONGRUENCY WITH THE ALBERTA PROGRAM: 
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we. 


‘ Shee se Oe ae ae ie a cece. SS 
eg ~.,. °- SUMMARY OF ‘KITS’ FOR ‘WHICH QUESTIONS WERE RAISED CONCERNING - 


. ay 7 ; . 
“4 e - ie - ny 


‘a4 .,. °° SUMMARY OF KITS FOR WHICH QUESTIONS WERE RAISED REGARDING 


ja Cae . : _-SCONGRUENCY WITH CURRENT SOCTAL STUDIES PRACTICE an 
ot : oo -_ , * Indicates, on were raised ae, 8 ; 
- CHA: TERISTICS Or SOUND PROGRAMS arial : | 7 
mp 12 3 4 5 6.7 8 9 10 11 12 taey 
a sijeetives! ate definite see pees , | a ae 
as ‘opposed to grandiose | and all in- * : 


clusive, 


* 


‘Social, science papeatet’ neces ‘are 
included in the skill‘ objective. 


’ the learner's present knowledge and x | 


_ A wide eanee of suggested subject mat- 
_. ter, learning activities and’ materials 
‘are provided from which the teacher 

may select. ; ") 


Content takes into account the inter- 
ests and information children have ~ 7 
_ from travel and mass media. . . 


Unit content is chosen in relation to-° 


. 
coe 


eres 
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Contemporary Acial ‘change and recent 
scholarly study are incorporated. . 


_.Current affairs is-an integral part ‘ 
of the unit. - a, 


| "Provision ie made for self and group 
; evaluation. 


‘Multiple viewpoints are presented. 


‘Provision is made for the sequential 


- Original data sources are utilized. 


“Inetructional methods designed for - 


' Tactics are designed to involve ieacae 
' understanding and skills. 
Attention is given to:/. 


‘= soctal action skills 


+ yse of commercial television’ 


Sctal aves ore eraineds es tk 


aN 


development of gkills. 


specific purpodes are utiliged. 


A variety of approaches and Bana 
ures are used. 
ers in processes leading to advanced — 


“ learning through community 
exper{ences 


and ‘integral part ‘ ‘of pa and somavenansive view 


. of Canada? 


» 2. Without predetermined criteria for. knowledge selgotibn, 3 
hos could “kit devetopers avoid overlap or imbelance in ie 
- ee. % 
‘ content coverage? ’ . ge . 
4 - ; e eo i 


Although the igetenacy of this post hoc content analyexe of the ey is. 


quaptionable, it is justified in terms of the oe axpeccarzone “ the pro- 


jqct. Parents and tive pabiice at large seem to. assume that as a result of this 
—_ we 
Canadian content project, students can be held accofintable for some logically 
ecgenteaas canadian content, Somewhere then, that content had’ to be specified. 
It would have been praterable ‘th t this task ae uridertaken prior to construc 
bo ee ap de oh Si ded ww Vee Fo ceca s ees ede ak : = - Bes & " 


tion of the thirteen kits. 
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’ At what “evel” (Canada, casional. Alberta) do the mete help ‘develop ie 


student perceptigns of eenatsae identity? ‘The fol lowing questions and gqner- 


-alizations have been selected From current literature on ‘Canadian congent- and 
) 


. are proposed by euthors as eeibetva for selecting significant Canadian’ content. 
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“this literature included varied sources such as the Symons Report on Canadj an 
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content (To Know buseives):, occasional papers of the Canada Studies Founda- 


tion, and, selected news paper /magazine articles. : * 5 
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“An examination of the charts. gives rise to the following questions: 


“ ‘1a: Have issues dealing with the future of Canada been 
r adequately covered? 
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4 2 Are - we justified in ignoring Canadians’ responsibilities 
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to tHose. outside our bordere: in a Canadian Gontent: 
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“Program? = :  f 
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LEVEL AT WHICH THE KITS HELP DEVELOP STUDENT — 

PERCEPTION OF CANADLAN IDENTITY 


who are we? oe . : 5 4 
sence) 


gwhore are we in time? 


i poe 


Where are we in space? 
(geography) 


“Where have we been? ~ 
(history) - 


Where are we going? 
; . (future) * 


What do we possess? 
(resources/talent) — 


eae a en es 


what are our responsi- 
bilities to ourselves? a 
— | (Canad{an) a’ oe 


What are our responsi- 
bilities to others? (— 
(Wor 1d) ea 
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on a3 oO 
3. Is it sound program pldnning to concentrate the major = * 


geography learnings ifthe elementary school? 


a In a country where. regional disparitjes play such.a 
prominent role {nJnational life can the absence of 
attention to regionalism be defended? | 


5. Should the topics of yrbanization and industrialization 


receive more emphasis in a Canadian Content, Program? 


. 6. Should, the external influences upon Canada 0 empraten 
‘es ? ; ‘ i . Poa ‘ 
‘4 bg ees a 
almost exclusively at the secondary school leyel? 
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225 ° FOR CONSIDERATION a 
a 2 Certain: patterns are evident in. the » preceeding charts. 
ofthe, number ‘of ueations raised at the elementary level | f 


may make: it, approptiiate to. hire overall editors for the 


division I and Il kits. ~ = °° a 


2. Should .a series of short "How To Do rt" booklets be 


written and be’ included within each kee. addressed to 


: se specific feoncerns. “We would ‘recommend the fol loving: 
— (a) Yow to integrate current, events within the on- 
* ‘ae "going antes. e . v | , 
’ “e” a : te : 


¢ (bd) How tg upilize commercial television “as an in- +42 


» 
Oo ne ee ere 


~tegral “part of tthe units. oe a re 


. ‘ oe 


(c) How t incorpordee: certain social. sctence ‘research 
2 | 


& ‘ 
/ 
methods such as surveyine, Antetviewing, polling, é 
a a, and document: analysis. 
oe ‘ / : ‘w ‘ . : a . 
| (a) How to “avelop, soeial action skills: ‘“~.°||OS 


These booklets shoyld be written for the non-specialist 
3 . social studies teacher at all grade ‘levels. 


3. Should ‘a technical pamphlet based on the pilot student 


£ . 
and. teacher reactions to the unit be included with each 
gl J . J 7 - | p , | . 
4. “shoul ie ae be produced before the questions posed 
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| eo : 7, “CANADIAN CONTENT EVALUATION No as oe or 
De iment of Elementary Education, " 


‘ 
5 
° » 
‘\ 
PAS 
ERIC. 


- Pilot Teachers*of Canadian 
Gontent Kits 


Don Massey and Walt Werner 


“le, gs 


~ The: ‘student survey iS’ concerned wi th pup41 interest in big kits. 


- or by March 25, 
_your convenience. 


oo Untversity of Alberta <r a ase 
. a EDMONTON, sais T6G -2G6 = oy oF 


February 22, 1977. 


s 


Pleasé find enclosed ‘two surveys: a 
(t) a teagher survey, and | 
+(2) a student survey,  ° _ = 


These surveys solicit teacher and studdnt sos akeians of the Canadian 
Content: Kits being piloted. These kits will be revised on the basis’of - 
your assessments. :Because Alberta Education will commit. substantia] 
funds: for the production and distribution of these kits throughout the ‘ 
province, your comments help assure that the: monte spent will ‘result ind. . a 
quality social suet programs . _* s : 


*Your reservations and recommendations about their ‘ictrueclonal quality 

will be treated confidentially. Do not judge the technical quality 
the quality of sound, drawing, and color) as ACCESS will be es 
for this after the piloting is completed. 


You 

may wish to discuss the questions with them, as ‘well as a that their 
concerhs will be used for jevesing the kits. 

Please return sie cies ecanna ns: immediately upon comptetion of the kit 

1977. ..A stamped and addressed envelope is enclosed for 

Call us at 432-5093 if you a any further information. 


Encl, : | : 
cc: Social Studies Consultants. | "oy 
Dr. D: Ledgerwood - *® a ( , 
' Mr. M.°Kowalchuk, 3 


Dr. K. Nixon 
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Pilot Student Questionnaires ° - ae 
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| Explain that the questions: are e about the things students did ioe: 
. ~ the study on Canada. The: teachers who constructed the unit .want student V- 


+ comments on how the materials may be ‘improved. , : 


‘Read each item to. the children; allowing enough time for them: to try 
| each: itd, The questions are not ‘speed tests. By administering the. test 
item-by+item theinstructions are rejnforced. Teachers are able to adjust 
* pace in keeping with the characteristics of the students being questioned. 
«By reading the directions aloud it is possible to reduce the influence of . - . $ 
reading: skills on’ the Survey results. . oe > % - ate oe 
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while learning about Canadian TONGS: 
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did white Ioannis about poeenern Canadian COMMUNE NESS: 
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